Memory Hierarchy

Memory Hierarchy

cache:architecturememory:hierarchyEach CPU in a Challenge/Onyx system accesses memory through a four-level hierarchy:

First-level instruction and data caches within the CPU chip provide the fastest access to recently-used data (the cache size depends on the microprocessor model).
A larger second-level cache on each CPU board stores recently-used instructions and data (this cache size depends on the CPU board model).
Main memory contains the current state of swapped-in processes.
Swapped-out virtual pages are kept in the swap partition on disk.

locality of referenceThere is a ratio of roughly 100:1 in access speeds between each level of this hierarchy. There is a large reward of execution speed for a program that maintains locality of reference, and so executes mostly out of cache. This is examined in more detail under "Reducing Cache Misses". At the other extreme, there is a large penalty of lost time for any program that causes pages to be swapped in and out of memory.

Cache Coherency Updates

cache coherencyEach CPU has two levels of cache that hold copies of memory data. Copies of the same data can exist in multiple caches at the same time. When a CPU writes to its cache memory, it broadcasts the fact on the processor bus. Other CPUs that have cached the same location mark their cached copies as invalid, so that if they need to refer to it again, they will reload the modified data.

This is a greatly oversimplified summary of a complicated protocol that ensures consistent, correct behavior of the multiple CPUs, even when they use the same memory areas. (For details on the subject, refer to one of the MIPS processor books listed in "Other Useful Books" on page xxiii.) Cache coherence is built into the hardware at a low level, and your program does not need to take any special steps to maintain it.